OSU-HEP-09-07 



Fermion Mass Hierarchy and 
New Physics at the TeV Scale 

S. Nandi 1 

Department of Physics and Oklahoma Center for High Energy Physics, 
Oklahoma State University, Stillwater, OK 74078 

Abstract. In this talk, I present a new framework to understand the long-standing fermion mass 
hierarchy puzzle. We extend the Standard Model gauge symmetry by an extra local U (1 )s symmetry, 
broken spontaneously at the electroweak scale. All the SM particles are singlet with respect to this 
[/(l)^. We also introduce additional flavor symmetries, U(1)f's, with flavon scalars F,, as well as 
vectorlike quarks and leptons at the TeV scale. The flavon scalars have VEV in the TeV scale. 
Only the top quark has the usual dimension four Yukawa coupling. EW symmetry breaking to all 
other quarks and leptons are propagated through the messenger field, S through their interactions 
involving the heavy vector-like fermions and S, as well as through their interactions involving the 
vector-like fermions and Ft. In addition the explaining the hierarchy of the charged fermion masses 
and mixings, the model has several interesting predictions for Higgs decays, flavor changing neutral 
current processes in the top and the b quark decays, decays of the new singlet scalars to the new Z' 
boson, as well as productions of the new vectorlike quarks. These predictions can be tested at the 
LHC. 
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INTRODUCTION 

Fermion mass hierarchy is a long-standing problem in particle physics. The charged 
fermion masses vary by five orders of magnitude, while the quark mixing angles vary 
by almost two orders of magnitude. There are two main approaches to understand this 
puzzle [1, 2, 3, 4, 5]. This hierarchy is caused by physics at the high scale (GUT scale 
or the Planck scale: so called Froggatt-Nielsen type mechanism; or this is caused by 
some new physics at the TeV scale. In this work, I present a new model for this second 
approach with new physics at the TeV scale [6]. 

What are the new physics possibilities at the TeV scale? Supersymmetry is highly 
motivated, and predicts new superpartners and the Higgs boson at the TeV scale. Extra 
dimensions are somewhat motivated, and predicts new Kaluza-Klein excitations at the 
TeV scale. Extra U{\) is somewhat string theory motivated, and predicts new gauge 
boson at the TeV scale. However these are all theory motivated. The experimental clues 
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so far are that the charged fermion masses are highly hierarchial, quark mixing angles 
are hierarchial, and FCNC processes are strongly suppressed. The natural question is 
what sort of new physics can explain these, and can be observed at the LHC. In this talk, 
I present one such possibility. 

In the SM, the Yukawa interactions of the fermions, in the mass basis, are parameter- 
ized by 

L = ydflihdiRH + y Ui qiLUiRH + h.c. ( 1 ) 

with m qi = y qi v, where v=< H >. Note that y t ~ 1, whereas the Yakawa couplings of 
all the light quarks and leptons, yb,yc,ys,ydiyu,ye,yn,y-i; are <C 1. Thus the top quark is 
directly connected to the EW symmetry breaking sector, and has dimension 4 Yukawa 
interactions. The lighter quarks are probably not directly connected to the EW symmetry 
breaking sector, They may be connected via some messenger fields. 

We know that the FCNC interactions among the quarks are highly suppressed. This 
hints at the existence of some sort of flavor symmetry. Furthermore, if we let all the 
SM fermions except q^ L , u^r and H carry nonzero flavor charges, then the dimension 4 
Yukawa couplings for the light quarks with H will be prevented. These flavor symmetries 
need to be global, and have to be spontaneously broken at the TeV scale. What sort of 
fields in addition to the SM we need to achieve this scenario? We shall see that one 
possibility is to have vectorlike quarks, Q and leptons at the TeV scale, and additional 
flavor symmetries, £7,(1)^. Again, since the light quarks and leptons are not directly 
connected to the EW symmetry breaking scale, we need a messenger field to achieve 
this, and a SM singlet complex scalar field with an extra U(l)s local symmetry will 
serve our purpose. Thus the new physics in our scenario will involve Q, S and Z . 

MODEL AND FORMALISM 

We extend the gauge symmetry of the SM model by a U(l)s local symmetry and Ui{\)p i 
global symmetries. All of the SM fermions are neutral with respect to U(l)s, while all 
of the SM fermions apart from the third generation quark doublet and right-handed 
top U3R are charged under the global Ui{\)p r We introduce a complex scalar field S 
which has charge 1 under U(l)s, is neutral under U^l)^, and is a SM singlet. We also 
introduce complex scalar fields F[, the "flavons", which have charges under U^l)^, are 
neutral under U(l)s, and are SM singlets. The Higgs field H is taken as neutral under 
U(l)s and Ui(l)p r We assume that the Ui{\)p i charges of the SM fermions are such that 
only the top quark has an allowed dimension 4 Yukawa interaction. 

The S field acquires a vev at the EW scale that spontaneously breaks the U(l)s 
symmetry. The pseudoscalar component of S is eaten to give mass to the U(l)s gauge 
boson Z ' . The field S acts as a messenger to both the flavor symmetry breaking and EW 
symmetry breaking. The Ui{\)p i symmetries are broken by the vev of the flavon scalar 
fields, Ft at the TeV scale. There are additional vectorlike fermions at the TeV scale 
charged under both U ( 1 ) 5 and U[ ( 1 ) p i . 

In this framework, the Yukawa interactions of the light fermions, after integrating out 
the heavy vectorlike fermions appear as higher dimensional operators in a hierarchial 



pattern given by 



w) (I) 1 (m) 



with similar expressions for the up sector. The observed fermion mass hierarchy and 
mixings are reproduced in powers of £ 

< s > 1 
M 7' 

which we call the "little" hierarchy. We can absorb the F/M dependence into field- 
dependent dimensionless complex couplings hy, where i, j are generation labels. The 
values of these couplings we will then take to be of order 1 . 

In the model we propose, the observed fermions mass hierarchy is generated from the 
following low energy effective interactions: 

if Yuk = h u 33 q 3L u 3R H+(^j (hi 3 q 3L d 3 RH + h u 22 q 2L U2RH + h u 23 q 2L u 3 RH 

J (h d 22 q 2L d 1R H + h d 23 q 2L d 3R H + h c 3 l 2 q 3L d 2R H 
+h u n q lL u 2R H + h 21 q 2L u iR H + h" 3 q lL u 3R H + h 3l q 3L ui R H) 

+ ( W J { h nViLUiRH + h d {q lL d lR H + hf 2 q lL d 2R H + h d 2l q 2L d m H 

+ hf 3 q lL d 3R H + h d 3l q 3L di R H^j + h.c. (2) 

where all the couplings are assumed to be of order 1 . 

Note that the above interactions are very similar to those proposed in reference [7, 8], 

except our interactions involve suppression by powers of (jpf) , instead of f^r) • 



Fit to Fermion Masses and CKM Mixing 

The gauge symmetry of our model is the usual SM symmetry, plus an additional U( l)s 
symmetry. The SM symmetry is broken spontaneously by the usual Higgs doublet, H at 
the EW scale. We assume that the extra U(l)s symmetry is also broken spontaneously 
at the EW scale by a SM singlet complex scalar field, S. The pseudoscalar part of 
the complex scalar field, S is absorbed by the Z' to get its mass. Thus after symmetry 
breaking, the remaining scalar fields are h and s. Parameterizing the Higgs doublet and 
singlet in the unitary gauge as 



with v ~ 174 GeV, and defining an additional small parameter 



P = 



M 



(4) 



we obtain, from Eqs. (2-4) the following mass matrices for the up and down quark sector: 



M u = 



'h u n e 6 
h u 2l £ A 
h» 3l £ A 



h\ 2 £ A 
h u 22 £ 2 
h» 2 e 2 



hf 3 E^ 
h\ 3 £ 2 

n 33 



'h d n e 6 
M d = | h d 2l e 6 



Mi* 



h d n £ e 

h 22 e 4 
h 32 £ 4 



h d 3 £ e 

h d 3 £ 2 



(5) 



The charged lepton mass matrix is obtained from Md by replacing the couplings /in- 
appropriately. Note that these mass matrices are the same as in Ref. [7], and as was 
shown there, good fits to the quark and charged lepton masses, as well as the CKM 
mixing angles are obtained by choosing £ ~ 0.15, and all the couplings /z,j of order one. 
To leading order in £, the fermion masses are given by 



(m t ,m c ,m u ) ~ (\h u 33 \, M^ 2 , Wi -h\ 2 h u 2X /h" 22 \e 6 ) v , 
(m h ,m s ,m d ) ~ (\hj 3 \£ 2 , \h\ 2 \£ A , \h d n |e 6 ) v , 
(m T , m M , m e ) ~ (\h f 33 \£ 2 , \h[ 2 \e A , \h e n |e 6 ) v , 



(6) 



while the quark mixing angles are 
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(7) 



Yukawa Interactions and FCNC 

Our model has flavor changing neutral current interactions in the Yukawa sector. 
Using Eqs. (1-4), the Yukawa interaction matrices Yf}, Y%, Y*, for the up and down 
sector, for h° and s° fields are obtained to be 

/h u n e 6 h u n e A h u 13 e A \ /h d n e 6 h d 2 e 6 h d 3 e 6 \ 

V2Y* = h u 2l e A h u 22 e 2 h u 23 e 2 , V2Y% = h d 2l e 6 h d 2 e A h d 3 £ A , (8) 
\h 3l £ A h 32 £ 2 h 33 / \h 3l £ 6 h d 2 £ A h 33 £ 2 J 

with the charged lepton Yukawa coupling matrix Yg obtained from Yd by replacing 

h d . -/,(. 



V2Y:= \4h u 21 e 3 p 2h u 22 ejl 2^ 3 ej3 , (9) 

\4h^£ 3 p 2h% 2 £j3 / 

/6h d n £ 5 p 6h d l2 £ 5 p 6/z? 3 £ 5 /3\ 

V2Y$= 6^,e 5 j8 4/4e 3 /3 4/ile 3 j3 I , (10) 



with the charged lepton Yukawa coupling matrix Y? obtained from Yj by replacing 

h d . -/,(. 
ij v 

There are several important features that distinguish our model from the proposal of 
Refs. [7, 8, 9]. i) Note, from Eqs.(5) and (8), in our model, the Yukawa couplings of h to 
the SM fermions are exactly the same as in the SM. This is because the fermion mass hi- 
erarchy in our model is arising from This is a distinguishing feature of our model 



from that proposed in [7, 8] where the Yukawa couplings of h are flavor dependent, be- 
cause the hierarchy there arises from (^jjf^j- ii) In our model, we have an additional 
singlet Higgs boson whose coupling to the SM fermions are flavor dependent as given 
in Eq. (9, 10). Again, this is because the hierarchy in our model arises from (jj^Pj- In 

particular, s° does not couple to the top quark, and its dominant fermionic coupling is 
to the bottom quark. This will have interesting phenomenological implications for the 
Higgs searches at the LHC. iii) We note from Eq. (5-8) that the mass matrices and the 
corresponding Yukawa coupling matrices for h are proportional as in the SM. Thus there 
are no flavor changing Yukawa interactions mediated by h. However, this is not true for 
the Yukawa interactions of the singlet Higgs as can be seen from Eqs. (5) and (9, 10). 
Thus s exchange will lead to flavor violation in the neutral Higgs interactions. 



Higgs Sector and Extra Z' 

The Higgs potential of our model, consistent with the SM and the extra U(\)s sym- 
metry, can be written as 

V(H,S) = -n]i(H^H)-ni(S^S) + X H (H^H) 2 + Xs(S^S) 2 + X H s(H^H)(S^S). (11) 

Note that after absorbing the three components of H in W ± and Z, and the pseu- 
doscalar component of S in Z', we are left with only two scalar Higgs, h° and s°. The 
squared mass matrix in the (h°,s°) basis is given by 

where a = v s /v. 



The mass eigenstates h and s can be written as 

h° = hcosO +5sin0, 

s° = -/zsine+scose, (13) 

where 9 is the mixing angle in the Higgs sector. 

In the Yukawa interactions discussed above, as well as in the gauge interactions 
involving the Higgs fields, the fields appearing are h° and s°, and these can be expressed 
in terms of h and s using Eq. (13). 

The mass of the Z' gauge boson is given by 

m\, = 2g 2 E v 2 s (14) 

Note that the Z' does not couple to any SM particles directly. Its coupling with the 
neutral scalar Higgs h (Z'hh coupling) is also zero. The Z' coupling to the SM particles 
will be only via dimension six or higher operators. Such couplings are generated by the 
vectorlike fermions in the model. 



PHENOMENOLOGICAL IMPLICATIONS: CONSTRAINTS, 
PREDICTIONS AND NEW PHYSICS SIGNALS 

Constraint on the mass of s: Experiments at LEP2 have set a lower limit of 114.4 
GeV for the mass of the SM Higgs boson. This is due to the nonobservation of the 
Higgs signal from the associated production e + e~ — > Zh. In our model, since the singlet 
Higgs can mix with the doublet h, there will be a limit for m s depending on the value 
of the mixing angle, 9. For cos 2 9 > 0.25, the bound of 114.4 applies also for m s [10]. 
However, s can be lighter if the mixing is small. 

Constraint on the mass of the Z'\ We have assumed that the extra U(l) symmetry 
in our model is spontaneously broken at the EW scale. But the corresponding gauge 
coupling, g£ is arbitrary and hence the mass of Z' is not determined in our model. 
However, very accurately measured Z properties at LEP1 put a constraint on the Z — Z' 
mixing to be ~ 10 3 or smaller [11, 12]. In our model, the Z' does not couple to any 
SM particle directly. Z — Z 1 mixing can take place at the one loop level with the new 
vectorlike fermions in the loop. The mixing angle is 

fl gzgE (m z \ 2 

0zz '~16^1mJ ' (15) 

where M is the mass of the vectorlike fermions with masses in the TeV scale. Even with 
g E ~ 1, we get 9 Z z> ~ 10~ 4 or less. Thus there is no significant bound for the mass of 
this Z' from the LEP1 [13]. 

Higgs signals: As can be seen from Eq. (8), the couplings of the doublet Higgs h to 
the SM fermions are identical to that in the SM, whereas the couplings of the singlet 
Higgs are flavor dependent. In particular, the singlet Higgs s does not couple to the top 
quark, whereas its coupling to (b, T;c,s,ii;u,d,e) involve the flavor dependent factors 
2, 2; 2, 4, 4; 6, 6, 6) respectively. This is, of course, in the limit of zero mixing between 



Higgs Branching Ratios, 6=0 
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FIGURE 1. Branching ratio of h -» 2x. Here a = 1 . 



h and s. Including the mixing, these factors will be modified by the appropriate mixing 
factors. Thus our model will be distinguished from the SM by the fact that the Higgs 
couplings, in general, will be fermion flavor dependent. 

Higgs decays: The couplings of the Higgs bosons h and s to the fermions and 
the gauge bosons can be obtained from Eqns. (8) and (9, 10). Because of the flavor 
dependency of the couplings of so (and hence for both h and s via mixing) to the 
fermions, the branching ratios (BR) for h to various final states are altered substantially 
from those in the SM. These branching ratios (BR) for h to the various final states are 
shown in Figs. 1, 2 for the values of the mixing angle, 0=0° and 26° [14]. 

For 0=0, these BR's are the same as for the SM. Note that for = 26°, the gg 
and 77 the BR's are enhanced substantially compared to the SM. This is due to drastic 
reduction for the bb mode due the almost cancelation in the corresponding coupling In 
particular, for 6 = 26°, the effect is quite dramatic. For a light Higgs (m^ around 115 
GeV), the usually dominant bb mode is highly suppressed and the 77 mode is enhanced 
by a factor of almost 10 compared to the SM. This is to be contrasted with the proposal 
of Refs. [7, 8] in which the h — > 77 mode is reduced by about a factor of 10. Thus the 
Higgs signal in this mode for a Higgs mass of ~ 1 14 — 140 GeV gets a big enhancement 
making its potential discovery via this mode much more favorable at the LHC. Such a 
signal may be observable at the Tevatron for a Higgs mass ~ 1 14 GeV as the luminosity 
accumulates, but would require about 10 fb 1 of data [15]. 

Another interesting effect is the Higgs signal via the WW* for the light higgs. In the 
SM, this mode becomes important for the Tevatron search staring at m/ 7 ~ 135 GeV, 
where the BR to WW* is approximate equal to that of bb. Currently Tevatron Run2 
experiments have excluded SM Higgs mass in the range of 160 to 170 GeV (where the 
BR to WW* is around 100 percent) for this mode. In our model, for 6 = 26° for example, 
note that this cross over between the WW* mode and the bb mode takes place sooner 
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FIGURE 2. Branching ratio of h -> 2x Here a = 1 . 



than 135 GeV. Thus the Tevatron will be more sensitive to lower mass range than in the 
SM, and will be able to exclude mass ranges much smaller than 160 GeV. 

Top quark physics: In the SM, t — > ch mode is severely suppressed with a BR 
~ 10 -14 [16]. In our model, as can be seen from Eqs.(8) and (9, 10), although t — > ch 
is zero at tree level, we have a large coupling for t — > cs ~ 2ej8. This gives rise to 
significant BR for the t — > cs mode for a Higgs mass of up to about 150 GeV. If the 
mixing between the h and s is substantial, both decay modes, t — ► cs and t — > c/z will 
have BR ~ 10~ 3 . With a very large Zt cross section , a t f ~ 10 3 pb at the LHC, this can 
be a major discovery mode for higgs bosons at the LHC. Observation of signals for two 
different Higgs masses will also show clear evidence for new physics beyond the SM. 

Z'physics: Our model has a Z' boson in the EW scale from the spontaneous breaking 
of the extra U(l)s symmetry. As discussed before, since the Z — Z' mixing is very small 
~ 10 -4 or less, its mass is not constrained by the very accurately measured Z properties 
at LER Its mass can be as low as few GeV from the existing constraints. This Z' does 
not couple to the SM particles with dimension 4 operators. It does couple to s at tree 
level via the sZ'Z' interaction. Thus it can be produced via the decay of s (or h if there 
is a substantial mixing between h and s). This gives an interesting signal for the Higgs 
decays, s — > Z'Z', h — > Z'Z' if allowed kinematically. 

Bj? — > /i + jU : In our model this decay gets a contribution from an FCNC interaction 
mediated by s-exchange. The amplitude for this decay is A ~ Ah^h^z 6 fi 2 ■ Taking 
/3 ~ e, A ~ 4/i22^22 £8 ' an d w i m m e couplings ^22 ' ^22 ~ 1 > we obtain the branching ratio, 
BR(B° S -> H + jl~) ~ 10~ 9 . Current experimental limit for this BR is 4.7 x 10~ 8 [11], and 
thus this decay could be observed soon at the Tevatron as the luminosity accumulates. 

Vectorlike fermions, productions and decays: Our model requires vectorlike quarks 
and leptons, both SU (2) doublets, Qi and singlets f/ ; and D ; , with masses at the TeV 
scale. These will be pair produced at high energy hadron colliders via strong interaction. 



For example, for a 1 TeV vectorlike quark, the production cross section at the LHC 
is ~ 60 fb [17]. We need several such vectorlike quarks for our model. So the total 
production cross section will be few hundred fb. These will decay to the light quarks of 
the same electric charge and Higgs bosons (h or s): Q — > qh,qs. Thus the signal will be 
two high px jets together with the final states arising from the Higgs decay. For a heavy 
Higgs, in the golden mode (h — > ZZ, s — > ZZ, this will give rise to two high pj jets plus 
four Z bosons. In the case of a light Z', the final state signal will be two high pj jets plus 
8 charged leptons in the final state (with each lepton pair having the invariant mass of 
theZ'). 

A CONCRETE MODEL 

We have constructed a concrete model giving rise to the phenomenological Lagrangian. 
In addition to the SM, the model has a U(l)s local symmetry, and the U^l)^ (i = 
1,2,3) global symmetries. The global symmetries are slightly broken explicitly in the 
Higgs potential so that there are no unwanted Goldstone bosons. In addition to the SM 
fermions, the model has a complex scalar, 5, the flavon fields F^s, and several vector-like 
quarks, both weak doublets and weak singlets. The details of their charge assignments 
under these symmetries, and how one obtains the interaction Lagrangian can be found 
in [6]. 

CONCLUSIONS 

We have presented a proposal in which only the top quark obtains its mass from the 
Yukawa interaction with the SM Higgs boson via dimension four operators. All the 
other quarks receive their masses from operators of dimension six or higher involving a 
complex scalar Higgs S whose vev is at the EW scale. The successive hierarchy of light 

quark masses is generated via the expansion parameter (j^Pj ~ £ 2 , where e = jj| ~ 0. 15. 

All the couplings of the higher dimensional operators are of order one. We are able to 
generate the appropriate hierarchy of fermion masses with this small parameter £. Since 
v s is at the EW scale, the physics of the new scale, M is at the TeV. Because of the 
new degree of freedom at the EW scale, we have an EW singlet neutral scalar s, which 
gives rise to interesting new physics signals which can be tested at the LHC and at the 
Tevatron. There are new scenarios for the Higgs decays and the top quark physics. The 
model has a light Z' which can be produced via the Higgs decays at the LHC, and can 
give rise invisible Higgs decays, displaced vertices for the Z' decays, or multilepton final 
states arise from the Z' decays, depending on the mass and lifetime of the Z'. We have 
presented a model in which an effective interaction given in Eq. (1) can be realized. This 
requires the existence of vectorlike quarks and leptons, both EW doublets and singlets, 
at the TeV scale. These can be probed at the LHC. Their decays give rise to final states 
with 4 Z's or 4 Z''s and interesting new physics signals at the LHC. 
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